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1. Type of Application 



This new application is for a(n) 

(check one applicable item below) 

^ Original (nonprovisional) 

□ Design 
□ Piant 

WARNING; Do not use this transmittal for a completion in the U.S. of an International Application under 35 
U.S.C. § 371 (cX4), unless the International Application is being filed as a divisional, continuation 
or continuation-in-part application. 

WARNING: Do not use this transmittal for the filing of a provisional application. 

NOTE: If one of the Mowing 3 items apply, then complete wd attach ADDED PAGES FOR NEW APPLICATION 
TRANSMITTAL WHERE BENEFIT OF A PRIOR U.S. APPLICATION CLAIMED and a NOTIFICATION 
IN PARENT APPLICATION OF THE FILING OF THIS CONTINUATION APPLICATION. 

□ Divisional. 

□ Continuation. 

□ Continuation-in-part (C-l-P). 

2. Benefit of Prior U.S. Application® (35 U.S.C. §§ 119(e), 120, or 121) 

NOTE: A nonprovisional application may claim an invention disclosed in one or more prior filed copending 
nonprovisional applications or copending international applications designating the United States of 
America. In order for a nonprovisional application to claim the benefit of a prior filed copending 
nonprovisional application or copending international application designating the United States of 
America, each prior application must name as an inventor at least one inventor named in the later filed 
nonprovisional application and disclose the named inventor's invention claimed in at least one claim 
of the later filed nonprovisional application in the manner provided by the first paragraph of 35 U.S.C. 
§ 112. Each prior application must also be: 

(?) An international application entitled to a filing date in accortfance with POT Article 11 and 
designating the United States of America; or 

(ii) Complete as set forth in § t.5T(b); or 

(tit) Entitled to a filing date as set forth in § 1.53(b) or § 1.53(d) and include the basic filing fee set 
forth in § 1.16; or 

(tv) Entitled to a filing date as set forth in § 1.53fo) and have paid therein the processing and retention 
fee set forth in § 1.210 within the time period set forth in § 1.53(1). 

37 C.F.R. § 1.78(a)(1). 

NOTE: if the new application being transmitted is a divisional, continuation or a continuation-in-part of a parent 
case, or where the parent case is an International Application which designated the U.S., or benefit 
of a prior provisional application is claimed, then check the following Hern and complete and attach 
ADDED PAGES FOR NEW APPLICATION TRANSMITTAL WHERE BENEFIT OF PRIOR U.S. APPLICA- 
TION^) CLAIMED. 

WARNING: If an application claims the benefit of the filing date of an earlier filed application under 35 U.S.C. 

§§ 120, 121 or 365(c), the 20-year term of that application will be based upon the filing date of 
the earliest U.S. application that the application makes reference to under 35 U.S.C. §§ 120, 121 
or 365(c). (35 U.S.C. § 154(a)(2) does not take into account for the determination of the patent 
term, any application on which priority is claimed under 35 U.S.C. §§ 119, 365(a) or 365$).) For 
a c-f-p application, applicant should review whether any claim in the patent that will issue is 
supported by an earlier application and, if not, the applicant should consider canceling the reference 
to the earlier filed application. The term of a patent is not based on a claim-by-claim approach. 
See Notice of April 14, 1995, 60 Fed. Reg. 20,195, at 20,205. 
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WARNING: When the last day of pendency of a provisional application falls on a Saturday, Sunday, or Federal 
holiday within the District of Columbia, any nonprovisional application claiming benefit of the 
provisional application must be filed prior to the Saturday, Sunday, or Federal holiday within the 
District of Columbia. See 37 C.F.R. § 1.78(a)(3). 

IS The new application being transmitted claims the benefit of prior U.S. applica- 
^ tion(s). Enclosed are ADDED PAGES FOR NEW APPLICATION TRANSMITTAL 
WHERE BENEFIT OF PRIOR U.S. APPLICATIONS) CLAIMED. 

3. Papers Enclosed 

A. Required for filing date under 37 C.F.R. § 1.53(b) (Regular) or 37 C.F.R. § 1.153 
(Design) Application 

Pages of specification 

. Pages of claims 

H Sheets of drawing 

WARNING: DO NOT submit original drawings. A high quality copy of the drawings should be supplied when 
filing a patent application. The drawings that are submitted to the Office must be on strong, white, 
smooth, and non-shiny paper and meet the standards according to § 1.84. If corrections to the 
drawings are necessary, they should be made to the original drawing and a high-quality copy of 
the corrected original dra wing then submitted to the Office. Only one copy is required or desired. 
For comments on proposed then-new 37 C.F.R § 1.84, see Notice of March 9, 1988 (1990 O.G, 
57-62). 

NOTE: "Identifying indicia, if provided, should include the application number or the title of the invention, 
inventor's name, docket number (tf any), and the name and telephone number of a person to calf if 
the Office is unable to match the drawings to the proper application. This information should be placed 
on the back of each sheet of drawing a minimum distance of 1.5 cm. (5/8 inch) down from the top 
of the page ..." 37 C.F.R. § 1.84(c)). 

(complete the following, if applicable) 

□ The enclosed drawing(s) are photograph(s), and there is also attached a 
"PETITION TO ACCEPT PHOTOGRAPH(S) AS DRAWING(S)." C7 C.F.R. 
§ 1.84(b). 

□ formal 
j8f informal 

B. Other Papers Enclosed 

2* Pages of declaration and power of attorney 
/ Pages of abstract 
Other 

4. Additional papers enclosed 

□ Amendment to claims 

□ Cancel in this applications claims _ before 

calculating the filing fee. (At least one original independent claim must be 
retained for filing purposes.) 

□ Add the claims shown on the attached amendment. (Claims added have 
been numbered consecutively following the highest numbered original 
claims.) 

□ Preliminary Amendment 

□ Information Disclosure Statement (37 C.F.R. § 1.98) 

□ Form PTO-1449 (PTO/SB/08A and 08B) 

□ Citations 

(New Application Transmittal [4-1] — page 3 of 11) 



□ Declaration of Biological Deposit 

□ Submission of "Sequence Listing," computer readable copy and/or amendment 
pertaining thereto for biotechnology invention containing nucleotide and/or 
amino acid sequence. 

□ Authorization of Attomey(s) to Accept and Follow Instructions from Representa- 
tive 

□ Special Comments 

□ Other 

5. Declaration or oath (including power of attorney) 

NOTE: A newly executed declaration is not required in a continuation or divisional application provided that 
the prior nonprovisional application contained a declaration as required, the application being Hied is 
by ail or fewer than all the inventors named in the prior application, there is no new matter in the 
application being filed, and a copy of the executed declaration filed in the prior application (showing 
the signature or an indication thereon that it was signed) is submitted. The copy must be accompanied 
by a statement requesting deletion of the names of personfs) who are not inventors of the application 
being filed. If the declaration in the prior application was filed under § 1.47, then a copy of that 
declaration must be Wed accompanied by a copy of the decision granting § 1.47 status or, if a nonsigning 
person under § 1.47 has subsequently joined in a prior application, then a copy of the subsequently 
executed declaration must be filed. See 37 C.F.R. §§ 1.63(d)(lH3). 

NOTE: A declaration filed to complete an application must be executed, identify the specification to which it 
is directed, identify each inventor by full name including family name and at feast one given name, without 
abbreviation together with any other given name or initial, and the residence, post office address and 
country or citizenship of each inventor, and state whether the inventor is a sole or joint inventor. 37 
C.F.R. § 1.G3(aH1H4). 

NOTE: "The inventorship of a nonprovisional application is that inventorship set fortfi in the oath or declaration 
as prescribed by § 1.62, except as provided for in § 1.53(d)(4) and § 1.63(d). If an oath or declaration 
as prescribed by§ 1.63 is not filed during the pendency of a nonprovisional application, the inventorship 
is that inventorship set forth in the application papers filed pursuant to § 1.53fi) t unless a petition under 
this paragraph accompanied by the fee set forth in § 1.17(f) is filed supplying or changing the name 
or names of the inventor or inventors." 37 C.F.R. § 1.41(a)(1). 

JSf Enclosed 

Executed by 

(check all applicable boxes) 

KC inventors). 

□ legal representative of inventors). 
37 C.F.R. §§ 1.42 or 1.43. 

□ joint inventor or person showing a proprietary « 
interest on behalf of inventor who refused to sign 
or cannot be reached. 

□ This is the petition required by 37 C.F.R. § 1 .47 and the statement 
required by 37 C.F.R. § 1.47 is also attached. See item 13 below 
for fee. 

□ Not Enclosed. 

NOTE: Where the filing is a completion in tfre US. of an International Application or where the completion of 
the U.S. application contains subject matter in addition to the International Application, the application 
may be treated as a continuation or continuation-in-part, as the case may be, utilizing ADDED PAGE 
FOR NEW APPLICATION TRANSMITTAL WHERE BENEFIT OF PRIOR U.S. APPUCATION CLAIMED. 

□ Application is made by a person authorized under 37 C.F.R. § 1.41(c) on 
behalf of all the above named inventors). 
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(The declaration or oath, along with the surcharge required by 37 C.F.R. § 1.16(e) 

can be filed subsequently). 



□ Showing that the filing is authorized. 

(not required unless called into question. 37 C.F.R. § 1.41(d)) 

6. Inventorship Statement 

WARNING: If the named inventors are each not the inventors of all the claims an explanation, including the 
ownership of the various claims at the time the last claimed invention was made, should be 
submitted. 

The inventorship for aii the claims in this application are: 
^ The same. 

or 

□ Not the same. An explanation, including the ownership of the various claims at 
the time the last claimed invention was made, 

□ is submitted. 

□ will be submitted. 

7. Language 

NOTE: An application including a signed oatf? or declaration may be fifed in a language other than English. 
An English translation of the non-English language application and the processing fee of $130,00 
required by 37 C.F.R. § 1.17QQ is required to be filed with the application, or within such time as may 
be set by the Office. 37 C.F.R. § 1.52($. 

JSf English 

□ Non-English 

□ The attached translation includes a statement that the translation is accu- 
rate. 37 C.F.R. § 1.52(d). 

8. Assignment . { 

^ An assignment of the invention to S^V^fl (r f Af>Q I 

^ is attached. A separate □ "COVER SHEET FOR ASSIGNMENT (DOCU- 
MENT) ACCOMPANYING NEW PATENT APPLICATION" or □ FORM PTO 
1595 is also attached. 

□ will follow. 

NOTE: "If an assignment & submitted with a new application, send two separate letters-one for the application 
and one for the assignment." Notice of May 4, 1990 (1114 O.G. 77-78). 

WARNING: A newly executed 'CERTIFICATE UNDER 37 C.F.R. § 3. 73(b)* must be filed when a continuation- 
in-part application is filed by an assignee. Notice of April 30, 1993, 1150 O.G. 62-64. 
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9. Certified Copy 

Certified copy(ies) of application® 



Country 


Appln. No. 


Filed 


Country 


Appln. No. 


Filed 


Country 


Appln. No. 


Filed 



from which priority is claimed 

□ is (are) attached. 

□ will follow. 

NOTE: The foreign application forming the basis for the claim for priority must be referred to in the oath or 
declaration. 37 C.F.R. § 1.55(a) and 1.63. 

NOTE: This item is for any foreign priority for which the application being filed directly relates. If any parent 
U.S. application or International Application from which this application claims benefit under 35 U.S.C. 
§ 120 is itself entitled to priority from a prior foreign application, then complete item 18 on the ADDED 
PAGES FOR NEW APPLICATION TRANSMITTAL WHERE BENEFIT OF PRIOR U.S. APPLICATION®) 
CLAIMED. 

10. Fee Calculation (37 C.F.R. § 1.16) 
A. J?f Regular application 



CLAIMS AS FILED 


Number filed 


Number Extra 




Rate 


Basic Fee 
37 C.F.R. § 1.16(a) 
$760.00 


Total 










Claims (37 C.F.R. 

§ 1.16(c)) 36 - 


20 = H, 


X 


$ 18.00 




Independent 

Claims (37 C.F.R. n 

§ 1.16(b)) i - 


3 = 6 


X 


$ 78.00 


tt>B-*t 


Multiple dependent claim(s), 
if any (37 C.F.R. § 1.16(d)) 




+ 


$260.00 









□ Amendment cancelling extra claims is enclosed. 



□ Amendment deleting multiple-dependencies is enclosed. 

□ Fee for extra claims is not being paid at this time. 

NOTE: if the fees for extra claims are not paid on filing they must be paid or the claims cancelled by amendment, 
prior to the expiration of the time period set for response by the Patent and Trademark Office in any 
notice of fee deficiency. 37 C.F.R. § 1.16(d). 

Filing Fee Calculation $ iSib-OG 

B. □ Design application 

($310.00—37 C.F.R. § 1.16(f)) 

Filing Fee Calculation $ 
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C- □ Plant application 

($480.00-37 C.F.R. § 1.16(g)) 

Filing fee calculation 

11. Small Entity Statements) 

^ Statement® that this is a filing by a small entity under 37 C.F.R. § 1 .9 and 1 .27 
is (are) attached. 

WARNING: "Status as a small entity must be specifically established in each application or patent in which 
the status is available and desired. Status as a small entity in one application or patent does not 
affect any other application or patent including applications or patents which are directly or 
indirectly dependent upon the application or patent in which the status has been established. The 
refiling of an application under § 1.53 as a continuation, division, or continuation-in-part (including 
a continued prosecution application under § 1.53(d)), or the filing of a reissue application requires 
a new determination as to continued entitlement to small entity status for the continuing or reissue 
application. A nonprovisiona! application claiming benefit under 35 U.S.C. § 1 19(e), 120, 121, or 
365(c) of a prior application, or a reissue application may rely on a statement Wed in the prior 
application or in the patent if the nonprovisional application or the reissue application includes a 
reference to the statement in the prior application or in the patent or includes a copy of the 
statement in the prior application or in the patent and status as a small entity is still proper and 
desired. The payment of the small entity basic statutory filing fee will be treated as such a reference 
for purposes of this section." 37 C.F.R. § 1.28(a)(2). 

WARNING: "Small entity status must not be established when the person or persons signing the . . . statement 
can unequivocally make the required self-certification. 9 M.P.E.P., § 509.03, 6th ed., rev. 2, July 
1996 (emphasis added). 

(complete the following, if applicable) 

□ Status as a small entity was claimed in prior application 
/ , filed on 



from which benefit 



is being claimed for this application under 

35U.S.C. § □ 119(e), 

□ 120, 

□ 121, 

□ 365(c), 

and which status as a small entity is still proper and desired. 
□ A copy of the statement in the prior application is included. 
Filing Fee Calculation (50% of A, B or C above) 

NOTE: Any excess of the full fee paid will be refunded if small entrtiy status is established and a refund request 
are filed within 2 months of the date of timely payment of a full fee. The two-month period is not 
extendable under § 1.136. 37 C.F.R. § 1.28(a)- 

12- Request for International-Type Search (37 C.F.R. § 1.104(d)) 

(complete, if applicable) 

□ Please prepare an international-type search report for this application at the time 
when national examination on the merits takes place. 
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13. Fee Payment Being Made at This Time 

□ Not Enclosed 

□ No filing fee is to be paid at this time. 

(This and the surcharge required by 37 C.F.R 5 1.16(e) can be paid 
subsequently.) 

*§Zf Enclosed 

Jgf Filing fee 

Recording assignment 

($40.00; 37 C.F.R. § 1.21(h)) 

(Gee a tt ached "GOVCR 0 1 I EC T ron 

A39 I QNMENT ACCOM P ANYING N E W (Lfj /i n 

APPL I CATION".) - $ ' V VU 

□ Petition fee for filing by other than all the 
inventors or person on behalf of the inventor 
where inventor refused to sign or cannot be 
reached 

($130.00; 37 C.F.R. §§ 1.47 and 1.1 7(i)) $ 

□ For processing an application with a 
specification in 

a non-English language 

($130.00; 37 C.F.R. §§ 1.52(d) and 1.1 7(k)) $ 

□ Processing and retention fee 

($130.00; 37 C.F.R. §§ 1.53(d) and 1.210) $ 

□ Fee for international-type search report 

($40.00; 37 C.F.R. § 1.21(e)) $ 



NOTE: 37 C.F.R. § 1.210 establishes a fee for processing and retaining any application that is abandoned for 
failing to complete the application pursuant to 37 C.F.R. § 1.530} and this, as well as the changes to 
37 C.F.R. §§ 1.53 and 1.73(a)(1), indicate that in order to obtain the benefit of a prior U.S. application, 
either the basic filing fee must be paid, or the processing and retention fee of § 1.21(1) must be paid, 
within 1 year from notification under § 53(9. ^7/7/2 /? 

Total fees enclosed $ <fr- 

14. Method of Payment of Fees 

^ Check in the amount of $ 



□ Charge Account No. in the amount of 

$ 

A duplicate of this transmittal is attached. 
NOTE: Fees should be itemized in such a manner that it is clear for which purpose the fees are paid. 37 C.F.R. 
§ 1.22p). 
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15. Authorization to Charge Additional Fees 

WARNING: If no fees am to be paid on filing, the following items should not be completed. 
WARNING: Accurately count claims, especially multiple dependent claims, to avoid unexpected high charges, 
if extra claim charges are authorized. 
Sf The Commissioner is hereby authorized to charge the following additional fees 
by this paper and during the entire pendency of this application to Account No. 

fif 37 C.F.R. § 1.16(a), (f) or (g) (filing fees) 

tSf 37 C.F.R. § 1.16(b), (c) and (d) (presentation of extra claims) 

NOTE: Because additional fees for excess or multiple dependent claims not paid on filing or on later presentation 
must only be paid or these claims cancelled by amendment prior to the expiration of the time period 
set for response by the PTO in any notice of fee deficiency (37 C.F.R. § 1. 16(d)), it might be best not 
to authorize the PTO to charge additional claim fees, except possibly when dealing with amendments 
after final action. 

37 C.F.R. § 1 .1 6(e) (surcharge for filing the basic filing fee and/or declaration 
on a date later than the filing date of the application) 
J£f 37 C.F.R. § 1.17(a)(1H5) (extension fees pursuant to § 1.136(a)). 
^jf 37 C.F.R. § 1.17 (application processing fees) 
NOTE: . A written request may be submitted in an application that is an authorization to treat any concurrent 
or future reply, requiring a petition for an extension of time under this paragraph for its timely submission, 
as incorporating a petition for extension of time for the appropriate length of time. An authorization to 
charge all required fees, fees under § 1.17, or all required extension of time fees will be treated as a 
constructive petition for an extension of time in any concurrent or future reply requiring a petition for 
an extension of time under this paragraph for its timely submission. Submission of the fee set forth in 
§ 1.17(a) will also be treated as a constructive petition for an extension of time in any concurrent reply 
requiring a petition for an extension of time under this paragraph for its timely submission. " 37 C.F.R. 
§ 1.136(a)(3). 

□ 37 C.F.R. § 1.18 (issue fee at or before mailing of Notice of Allowance, 
pursuant to 37 C.F.R. § 1.311(b)) 

NOTE: Where an authorization to charge the issue fee to a deposit account has been filed before the mailing 
of a Notice of Allowance, the issue fee will be automatically charged to the deposit account at the time 
of mailing the notice of allowance. 37 C.F.R. § 1.311(b). 

NOTE: 37 C.F.R. § 1.28(b) requires "Notification of any change in status resulting in loss of entitlement to small 
entity status must be filed in the application . . . prior to paying, or at the time of paying, . . .the issue 
fee. . . " From the wording of 37 C.F.R. § 1.28(b), (a) notification of change of status must be made 
even if the fee is paid as "other than a small entity" and (b) no notification is required if the change 
is to another smalt entity. 
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16. Instructions as to Overpayment 

NOTE: *. . . Amounts of twenty-five dollars or less will not be returned unless specifically requested within 
a reasonable time, nor will the payer be notified of such amounts; amounts over twenty-five dollars may 
be returned by check or, if requested, by credit to a deposit account" 37 C.F.R. § 1.26(a). 

J5f Credit Account No. ^JAlMM. 

□ Refund 




JRE OF PRACTITIONER 



{type or print name of attorney) 



P.O. Address 



Customer No. >S&0 0*1' & forty 
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X 



Incorporation by reference of added pages 

(check the following item if the application in this transmittal claims the benefit of 
prior U.S. applications) (including an international application entering the U.S. 
stage as a continuation, divisional or C-/-P application) and complete and attach 
the ADDED PAGES FOR NEW APPLICATION TRANSMITTAL WHERE BENEFIT OF 
PRIOR U.S. APPLICATIONS) CLAIMED) 

Plus Added Pages for New Application Transmittal Where Benefit of Prior U.S. 
r ^ Application® Claimed £- 



Number of pages added . 



Plus Added Pages for Papers Referred to in Item 4 Above 

Number of pages added 



□ Plus added pages deleting names of inventors) named in prior application(s) 
who is/are no longer inventor{s) of the subject matter claimed in this application. 

Number of pages added ■ 

□ Plus "Assignment Cover Letter Accompanying New Application" 

Number of pages added . ■ 

□ Statement Where No Further Pages Added 

(if no further pages form a part of this Transmittal, then end this Transmittal with 
this page and check the following item) 

□ This transmittal ends with this page. 
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Practitioner's Docket No. 7 7 "77// ^ PATENT 

ADDED PAGES FOR APPLICATION TRANSMITTAL WHERE BENEFIT OF 
PRIOR UJS. APPLICATION^) CLAIMED 

NOTE: See 37 C.F.R. § 1.78. 

17. Relate Back 

WARNING: If an application daims the benefit of the filing date of an earlier filed application under 35 U.S.C. 

§§ 120, 121 or 365(c), the 20-year term of that application will be based upon the filing date of 
the earliest U.S. application that the application makes reference to under 35 U.S.C. §§ 120, 121 
or 365(c). (35 U.S.C. § 154(a)(2) does not take into account, for the determination of the patent 
term, any application on which priority is claimed under 35 U.S.C. §§ 1 19, 365fe) or 365(b).) For 
a c-hp application, applicant should review whether any claim in the patent that will issue is 
supported by an earlier application and, if not, the applicant should consider canceling the reference 
to the earlier filed application. The term of a patent is not based on a claim-by-ctaim approach. 
See Notice of April 14, 1995, 60 Fed. Reg. 20,195, at 20,205. 

(complete the following, if applicable) 

j^f Amend the specification by inserting, before the first line, the following sentence: 

A. 35 U.S.C. § 119(e) 

NOTE: "Any nonprovisional application claiming the benefit of one or more prior filed copending provisional 
applications must contain or be amended to contain in the first sentence of the specification following 
the title a reference to each such prior provisional application, identifying it as a provisional application, 
and including the provisional application number (consisting of series code and serial number). "37 C.F.R. 
§ 1.78(a)(4)- 

J3( This application claims the benefit of U.S. Provisional Application® No(s).: 
APPUCATION NO(S).: HUNG DATE 
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B. 35 U.S.C. §§ 120, 121 and 365(c) 

NOTE: "Except for a continued prosecution application filed under § 1.53(d), any nonprovisional application 
claiming the benefit of one or more prior filed copending nonprovisional applications or international 
applications designating the United States of America must contain or be amended to contain in the 
first sentence of the specification following the title a reference to each such prior application, identifying 
it by application number (consisting of the series code and serial number) or international application 
number and international filing date and indicating the relationship of the applications. . . . Cross- 
references to other related applications may be made when appropriate." (See § 1.14(a)). 37 C.F.R. 
§ 1.78(a)(2)- 

□ "This application is a 

□ continuation 

□ continuation-in-part 

□ divisional 

of copending application® 

□ application number 0 / filed on " 

□ International Application _ filed on 

and which designated the U.S." 

NOTE: The proper reference to a prior filed PCT application that entered the U.S. national phase is the U.S. 

serial number and the filing date of the PCT application that designated the U.S. 
NOTE' (V Where the application being transmitted adds subject matter to the International Application, then 

the filing can be as a continuation-in-part or (2) if it is desired to do so for other reasons then the filing 

can be as a continuation. 

NOTE: The deadline for entering the national phase in the U.S. for an international application was clarified 
in the Notice of April 28, 1987 (1079 O.G. 32 to 46) as follows: 

"The Patent and Trademark Office considers the International application to be pending until the 22nd 
month from the priority date if the United States has been designated and no Demand for International 
Preliminary Examination has been filed prior to the expiration of the 19th month from the priority date 
and until the 32nd month from the priority date if a Demand for International Preliminary Examination 
which elected the United States of America has been filed prior to the expiration of the 19th month 
from the priority date, provided that a copy of the international application has been communicated 
to the Patent and Trademark Office within the 20 or 30 month period respectively. If a copy of the 
international application has not been communicated to the Patent and Trademark Office within the 
20 or 30 month period respectively, the international application becomes abandoned as to the United 
States 20 or 30 months from the priority date respectivley. These periods have been placed in the rules 
as paragraph (h) of § 1.494 and paragraph (t) of § 1.495. A continuing application under 35 U.S.C. 365(c) 
and 120 may be filed anytime during the pendency of the international application." 

□ "The nonprovisional application designated above, namely application 

_____ / m , filed , claims the benefit of 

U.S. Provisional Application(s) No(s).: 



APPLICATION NO(S).: FILING DATE 




- 

□ Where more than one reference is made above, please combine all references 
into one sentence. 
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18. Relate Back— 35 U.S.C. § 119 Priority Claim for Prior Application 

The prior U.S. application®, including any prior International Application designating the 
U.S., identified above in item 17B, in turn itself claim(s) foreign priority(ies) as follows: 

Country Appln. no. Filed on 

The certified copyfies) has (have) 

□ been filed on , in prior application 0 / which was 

filed on ~ 

□ is (are) attached. 

WARNING: The certified copy of the priority application that may have been communicated to the PTO by 
the International Bureau may not be relied on without any need to file a certified copy of the priority 
application in the continuing application. This is so because the certified copy of the priority 
application communicated by the International Bureau is placed in a folder and is not assigned 
a U.S. serial number unless the national stage is entered. Such folders are disposed of if the national 
stage is not entered. Therefore, such certified copies may not be available if needed later in the 
prosecution of a continuing application. An alternative would be to physically remove the priority 
documents from the folders and transfer them to the continuing application. The resources required 
to request transfer, retrieve the folders, make suitable record notations, transfer the certified copies, 
enter and make a record of such copies in the Continuing Application are substantial. Accordingly, 
the priority documents in folders of international applications that have not entered the national 
stage may not be relied on. Notice of April 28, 1987 (1079 O.G. 32 to 46). 

19. Maintenance off Copendency of Prior Application 

NOTE: The PTO finds it useful if a copy of the petition filed in the prior application extending the term for 
response is filed with the papers constituting the filing of the continuation application. Notice of 
November 5, 1985 (1060 O.G. 27). 

A. □ Extension of time in prior application 

(This item must be completed and the papers filed In the prior application, 
if the period set in the prior application has^mn.) 

□ A petition, fee and response extends the term in the: pending prior application 
until — 

□ A copy of the petition filed in prior application is attached; 

B. □ Conditional Petition for Extension of Time in Prior Application 

(complete this item, if previous item not applicable) 

□ A conditional petition for extension of time is being filed in the pending prior 
application. 

□ A copy of the conditional petition filed in the prior application is attached. 
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20. Further Inventorship Statement Where Benefit of Prior Application!*) 
Claimed 

(complete applicable item (a), (b) and/or (c) below) 

(a) rhis application discloses and claims only subject matter disclosed in the prior 
application whose particulars are set out above and the inventor(s) in this 
application are 

ihe same. 

□ less than those named in the prior application. It is requested that the 
following inventor(s) identified for the prior application be deleted: 

(type name(s) of inventorfs) to be deleted) 

(b) □ This application discloses and claims additional disclosure by amendment and 

a new declaration or oath is being filed. With respect to the prior application, 
the inventor(s) in this application are 

□ the same. 

□ the following additional inventor® have been added: 

(type nawe(s) of inventors) to be added) 

(c) The inventorship for all the claims in this application are 

the same. 

□ not the same. An explanation, including the ownership of the various claims 
at the time the last claimed invention was made 

□ is submitted. 

□ will be submitted. 
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21. Abandonment of Prior Application (if applicable) 

□ Please abandon the prior application at a time while the prior application is 
pending, or when the petition for extension of time or to revive in that application 
is granted, and when this application is granted a filing date, so as to make this 
application copending with said prior application. 
NOTE* According to the Notice of May 13, 7983 (103, JMOG 6-75, the filing of a continuation or continuation-in- 
part application is a proper response with respect to a petition for extension of time or a petition to 
revive and should include the express abandonment of the prior application conditioned upon the 
granting of the petition and the granting of a filing date to the continuing application. 

22. Petition for Suspension of Prosecution for the Time Necessary to 
File an Amendment 

WARNING: The claims of a new application may be finally rejected in the first Office action in those situations 
where (A) the new application is a continuing application of, or a substitute for, an earlier application, 
and (B) all the claims of the new application (1) are drawn to the same invention claimed in the 
earlier application, and (2) would have been properly finally rejected on the grounds of art of record 
in the next Office action if they had been entered in the earlier application." M.RER, § 706.07(b), 
7th ed. 

NOTE: Where it is possible that the claims on file wilt give rise to a first action final for this continuation application 
and for some reason an amendment cannot be filed promptly (e.g., experimental data is being gathered) 
it may be desirable to file a petition for suspension of prosecution for the time necessary. 

(check the next item, if applicable) 

□ There is provided herewith a Petition To Suspend Prosecution for the Time 
Necessary to File An Amendment (New Application Filed Concurrently) 

23. Small Entity (37 C.F.R. § 1.28(a)) 

□ Applicant has established small entity status by the filing of a statement in parent 
application / on 

□ A copy of the statement previously filed is included. 
WARNING: See 37 C.F.R. § 1.28(a)- 

WARNING: "Smalt entity status must not be established when the person or persons signing the. . . statement 
can unequivocally make the required self-certification." M.P.E.P., § 509.03, 7th ed. (emphasis 
added). 

24. NOTIFICATION IN PARENT APPLICATION OF THIS FILING 

□ A notification of the filing of this 
(check one of the following) 

□ continuation 

□ continuation-in-part 

□ divisional 

is being filed in the parent application, from which this application claims priority under 35 
U.S.C. § 120. 

(Added Pages for Application Transmittal Where Benefit of Prior US. Applications) Claimed 
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verified statement is directed. 

Steven G. Apel 

Name Of Inventor/Assignee 




Page 2 of 2 



AUDIENCE SURVEY SYSTEM, AND SYSTEMS AND METHODS FOR 
COMPRESSING AND CORRELATING AUDIO SIGNALS 

FIELD OF THE INVENTION 

5 The invention relates to a method and system for automatically identifying which of a 

number of possible audio sources is present in the vicinity of an audience member. This is 
accomplished through the use of audio pattern recognition techniques. A system and method is 
disclosed that employs small portable monitoring units worn or carried by people selected to 
form a panel that is representative of a given population. Audio samples taken at regular 

1 0 intervals are compressed and stored for later comparison with reference signals collected at a 
central site. This allows a determination to be made regarding which broadcast audio signals 
each survey member is listening to at different times of day. An automatic survey of listening 
preferences can then be conducted. 

DISCUSSION OF THE PRIOR ART 

1 5 Radio and television surveys have been conducted for many years to determine the 

relative popularity of programs and broadcast stations. This information is necessary for a 
number of reasons including the determination of advertising price structure and deciding if 
certain programs should be continued or canceled. One of the most common methods for 
performing these surveys is for survey members to manually record the radio and television 

20 stations that they listen to and watch at various times of day. The maintaining of these manual 
logs is cumbersome and inaccurate. Additionally, transferring the information in the logs to an 
automated system represents an additional time consuming process. 

Various systems have been developed that provide a degree of automation to conducting 
these surveys. In a typical semiautomatic survey system an electronic device records which 

25 television station is being viewed in a survey member's home. The survey member may 



optionally enter the number of people who are viewing the program. These data are 
electronically transferred to a central location where survey statistics are compiled. 

Automatic survey systems have been devised that substantially improve efficiency. 
Many of the methods used involve the injection of a coded identification signal within the audio 
5 or video. There are several problems with these so-called active identification systems. First, 
each broadcaster must cooperate with the survey organization by installing the coding equipment 
in its broadcast facility. This represents an additional expense and complication to the 
broadcaster that may not be acceptable. The use of identification codes can also result in audio 
or video artifacts that are objectionable to the audience. An active encoding system is described 
10 by Best et al. in U.S. Patent 4,876,617. Best employs two notch filters to remove narrow 

frequency bands from the audio signal. A frequency shift keyed signal is then injected into these 
notches to carry the identification code. Codes are repeatedly inserted into the audio when there 
is sufficient signal energy to mask the codes. However, when the injection level of the code is 
sufficient to assure reliable decoding it is perceptible to listeners. Conversely, when the code 
1 5 injection level is reduced to become imperceptible decoding reliability suffers. Best has 

improved on this invention as taught in U.S. Patent 5,1 13,437. This system uses several sets of 
code frequencies and switches among them in a pseudo-random manner. This reduces the 
audibility of the codes. 

Fardeau et al. describe a different type of system in U.S. Patent 5,574,962 and U.S. Patent 
20 5,581,800 where the energy in one or more frequency bands is modulated in a predetermined 
manner to create a coded message. A small body-worn (or carried) device receives the encoded 
audio from a microphone and recovers the embedded code. After decoding, the identification 
code is stored for later transfer to a central computer. The problem remains that all broadcast 
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stations to be detected by the system must be persuaded to install code generation and insertion 
equipment in their audio feeds. 

Broughton et al. describe a video signaling method in U.S. Patent 4,807 ,031 that encodes 
a message by modulating the relative luminance of the two fields comprising a video frame. 
5 While intended for use in interactive television, this method can also be used to encode a channel 
identification code. An obvious limitation is that this method cannot be used for radio 
broadcasts. Additionally, the television broadcast equipment must be altered to include the 
identification code insertion. 

Passive signal recognition techniques have been developed for the identification of 
□10 prerecorded audio and video sources. These systems use the features of the signal itself as the 
identification key. The unknown signal is then compared with a library of similarly derived 
features using a pattern recognition procedure. One of the earliest works in this area is presented 
by Moon et al. in U.S. Patent 3,919,479. Moon teaches that correlation functions can be used to 
identify audio segments by matching them with replicas stored in a database. Moon also 
15 describes the method of extracting sub-audio envelope features. These envelope signals are 
y;l more robust than the audio itself, but Moon's approach still suffers from sensitivity to distortion 
and speed errors, 

A multiple stage pattern recognition system is described by Kenyon et al. in U.S. Patent 
4,843,562. This method uses low-bandwidth features of the audio signal to quickly determine 
20 which patterns can be immediately rejected. Those that remain are subjected to a high-resolution 
correlation with time warping to compensate for speed errors. This system is intended for use 
with a large number of candidate patterns. The algorithms used are too complex to be used in a 
portable survey system. 
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Another representative passive signal recognition system and method is disclosed by 
Lamb et al. in U.S. Patent 5,437,050. Lamb performs a spectrum analysis based on the 
semitones of the musical scale and extracts a sequence of measurements forming a spectrogram. 
Cells within this spectrogram are determined to be active or inactive depending on the relative 
5 power in each cell. The spectrogram is then compared to a set of reference patterns using a 
logical procedure to determine the identity of the unknown input. This technique is sensitive to 
speed variation and even small amounts of distortion. 

Kiewit et al. have devised a system specifically for the purpose of conducting automatic 
audience surveys as disclosed in U.S. Patent 4,697,209. This system uses trigger events such as 
QlO scene changes or blank video frames to determine when features of the signal should be 
4= collected. When a trigger event is detected, features of the video waveform are extracted and 
\Z stored along with the time of occurrence in a local memory. These captured video features are 
periodically transmitted to a central site for comparison with a set of reference video features 
from all of the possible television signals. The obvious shortcoming of this system is that it 
U 15 cannot be used to conduct audience surveys of radio broadcasts. 

=J:| The present invention combines certain aspects of several of the above inventions, but in 

a unique and novel manner to define a system and method that is suited to conducting audience 
surveys of both radio and television broadcasts. 



20 SUMMARY OF THE INVENTION 

It is an objective of the present invention to provide a method and apparatus for 
conducting audience surveys of radio and television broadcasts. This is accomplished using a 
number of body-worn portable monitoring units. These units periodically sample the acoustic 
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environment of each survey member using a microphone. The audio signal is digitized and 
features of the audio are extracted and compressed to reduce the amount of storage required. 
The compressed audio features are then marked with the time of acquisition and stored in a local 
memory. 

5 A central computer extracts features from the audio of radio and television broadcast 

stations using direct connection to a group of receivers. The audio is digitized and features are 
extracted in the same manner as for the portable monitoring units. However, the features are 
extracted continuously for all broadcast sources in a market. The feature streams are 
compressed, time-marked and stored on the central computer disk drives, 
□l 0 When the portable monitoring units assigned to survey members are not being worn (or 

■ carried), they are stored in docking stations that recharge the batteries and also provide modems 

; ^ and telephone access. On a daily basis, or every several days, the central computer interrogates 
j=l the docked portable monitoring unit using the modem and transfers the stored feature packets to 
the central computer for analysis. This is done late at night or early in the morning when the 
1 5 portable monitoring unit is not in use and the phone line is available. 
M3 In addition to transferring the feature packets, the current time marker is transferred from 

the portable monitoring unit to the central computer. By comparing the current time marker with 
the time marker transferred during the last interrogation the central computer can determine the 
apparent elapsed time as seen by the portable monitoring unit. The central computer then makes 
20 a similar calculation based on the absolute time of interrogation and the previous interrogation 
time. The central computer can then perform the necessary interpolations and time translations 
to synchronize the feature data packets received from the portable monitoring unit with feature 
data stored in the central computer. 
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By comparing the audio feature data collected by a portable monitoring unit with the 
broadcast audio features collected at the central computer site, the system can determine which 
broadcast station the survey member was listening to at a particular time. This is accomplished 
by computing cross-correlation functions for each of three audio frequency bands between the 
5 unknown feature packet and features collected at the same time by the central computer for many 
different broadcast stations. The fast correlation method based on the FFT algorithm is used to 
produce a set of normalized correlation values spanning a time window of approximately six 
seconds. This is sufficient to cover residual time synchronization errors between the portable 
monitoring unit and the central computer. The correlation functions for the three frequency 
□1 0 bands will each have a value of +1 .0 for a perfect match, 0.0 for no correlation, and -1 .0 for an 
exact opposite. These three correlation functions are combined to form a figure of merit that is a 
three dimensional Euclidean distance from a perfect match. This distance is calculated as the 
square root of the sum of the squares of the individual distances, where the individual distance is 
equal to (1 .0 - correlation value). In this representation, a perfect match has a distance of zero 
U 15 from the reference pattern. In an improved embodiment of the invention the contributions of 
l £l each of the features is weighted according to the relative amplitudes of the feature waveforms 

stored in the central computer database. This has the effect of assigning more weight to features 
that are expected to have a higher signal-to-noise ratio. 

The minimum value of the resulting distance is then found for each of the candidate 
20 patterns collected from the broadcast stations. This represents the best match for each of the 
broadcast stations. The minimum of these is then selected as the broadcast source that best 
matches the unknown feature packet from the portable monitoring unit. If this value is less than 
a predetermined threshold, the feature packet is assumed to be the same as the feature data from 
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the corresponding broadcast station. The system then makes the assertion that the survey 
member was listening to that radio or television station at that particular time. 

By collecting and processing these feature packets from many survey members in the 
context of many potential broadcast sources, comprehensive audience surveys can be conducted. 
5 Further, this can be done faster and more accurately than was possible using previous methods. 

DESCRIPTION OF THE DRAWINGS 

The features, objects, and advantages of the present invention will become more 
apparent from the detailed description set forth below when taken in conjunction with the 
CH 0 following drawings: 

4* Figure 1 illustrates the functional components of the invention and how they interact to 

; sl l function as an audience measurement system. Audience survey panel members wear portable 
monitor units that collect samples of audio in their environment. This includes audio signals 
from broadcast radio and television receivers. The radio and television broadcast signals in a 
it? 15 survey market are also received by a set of receivers connected to a central computer. Audio 
y:l features from all of the receivers are recorded in a database on the central computer. When not 
in use, portable monitor units are placed in docking stations where they can be interrogated by 
the central computer via dialup modems. Audio feature samples transferred from the portable 
monitor units are then matched with audio features of multiple broadcast stations stored in the 
20 database. This allows the system to determine which radio and television programs are being 
viewed or heard by each panel member. 

Figure 2 is a block diagram of a portable monitor unit. The portable monitoring unit 
contains a microphone for gathering audio. This audio signal is amplified and lowpass filtered to 
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restrict frequencies to a little over 3 kHz. The filtered signal is then digitized using an analog to 
digital converter. Waveform samples are then transferred to a digital signal processor. A low- 
power timer operating from a separate lithium battery activates the digital signal processor at 
intervals of approximately one minute. It will be understood by those skilled in the art that the 
5 digital processor can collect the samples at any period interval, and that use of a one-minute 
period is a matter of design choice and should not be considered as limiting of the scope of the 
invention. The digital signal processor then reads samples from the analog to digital converter 
and extracts features from the audio waveform. The audio features are then compressed and 
stored in a non- volatile memory. Compressed feature packets with time tags are later transferred 

Ql 0 through a docking station to the central computer. A rechargeable battery is also included. 

Figure 3 shows the three frequency bands that are used for feature extraction in a 

Til particularly preferred embodiment of the present invention. The energy in each of these three 
frequency bands is sampled approximately ten times per second to produce feature waveforms. 
Figure 4 illustrates the major components of the central computer that continuously 

U 15 captures broadcast audio from multiple receivers and matches feature packets from portable units 
with possible broadcast sources. A set of audio amplifiers and lowpass antialias filters provide 
appropriate gain and restrict the audio frequencies to a little over 3 kHz. A channel multiplexer 
rapidly scans the filter outputs and transfers the waveforms sequentially to an analog to digital 
converter producing a multiplexed digital time series. A digital signal processor performs a 
20 spectrum analysis and produces energy measurements of each of three frequency bands from 
each of the input channels. These feature samples are then transferred to a host computer and 
stored for later comparison. The host computer contains a bank of modems that are used to 
interrogate the portable monitor units while they are docked. Feature data packets are transferred 
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from the portable units during this interrogation. One or more digital signal processors are 
connected to the host computer to perform the feature pattern recognition process that identifies 
which broadcast channel, if any, matches the unknown feature packets from the portable 
monitoring units. 

5 Figure 5 is a block diagram of the docking station for the portable monitor unit. The 

docking station contains four components. The first component is a data interface that connects 
to the portable unit. This interface may include an electrical connection or an infrared link. The 
data interface connects to a modem that allows telephone communication and transfer of data. A 
battery charger in the docking station is used to recharge the battery in the portable unit. A 
;110 modular power supply is included to provide power to the other components. 
: = Figure 6 illustrates an expanded survey system that is intended to operate in multiple 

" cities or markets, A wide area network connects a group of remotely located signal collection 
systems with a central site. Each of the signal collection systems captures broadcast audio in its 
region and stores features. It also interrogates the portable monitoring units and gathers the 
1 5 stored feature packets. Data packets from the remote sites are transferred to the central site for 
Uj processing. 

Figure 7 is a flow chart of the audio signal acquisition strategy for the portable 
monitoring units. The portable monitoring units activate periodically and compute features of 
the audio in the environment. If there is sufficient audio power the features are compressed and 
20 stored. 

Figure 8 is a flow chart of procedures used to collect and manage audio features received 
at central collection sites. This includes the three separate processes of audio collection, feature 
extraction, and deletion of old feature data. 
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Figure 9 is a flow chart of the packet identification procedure. Packets are first 
synchronized with the database. Corresponding data blocks from broadcast audio sources are 
then matched to find the minimum weighted Euclidean distance to the unknown packet. If this 
distance is less than a threshold, the unknown packet is identified as matching the broadcast. 
5 Figure 1 0 is a flow chart of the pattern matching procedure. Unknown feature packets 

are first zero padded to double their length and then correlated with double length feature 
segments taken from the reference features on the central computer. The weighted Euclidean 
distance is then computed from the correlation values and the relative amplitudes of the features 
stored in the reference patterns. 
% 0 Figure 1 1 illustrates the process of averaging successive weighted distances to improve 

S the signal-to-noise ratio and reduce the false detection rate. This is an exponential process where 
old data have a smaller effect than new data. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

1 1 5 The audience measurement system according to the invention consists of a potentially 

large number of body-worn portable collection units 4 and several central computers 7 located in 
various markets. The portable monitoring units 4 periodically sample the audio environment and 
store features representing the structure of the audio presented to the wearer of the device. The 
central computers continuously capture and store audio features from all available broadcast 
20 sources 1 through direct connections to radio and television receivers 6. The central computers 7 
periodically interrogate the portable units 4 while they are idle in docking stations 10 at night via 
telephone connections and modems 9. The sampled audio feature packets are then transferred to 
the central computers for comparison with the broadcast sources. When a match is found, the 
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presumption is that the wearer of the portable unit was listening to the corresponding broadcast 
station. The resulting identification statistics are used to construct surveys of the listening habits 
of the users. 

In typical operation, the portable monitoring units 4 compress the audio feature samples 
5 to 200 bytes per sample. Sampling at intervals of one minute, the storage requirements are 200 
bytes per minute or 12 kilobytes per hour. During quiet intervals, feature packets are not stored. 
It is estimated that about 50 percent of the samples will be quiet. The average storage 
requirement is therefore about 144 kilobytes per day or approximately 1 Megabyte per week. 
The portable monitoring units are capable of storing about one month of compressed samples. 
Q'10 If the portable monitoring units are interrogated daily, approximately one minute will be 

:s |;* required to transfer the most recent samples to a central computer or collection site. The number 
of modems 9 required at the central computer 7 or collection site 33 depends on the number of 
portable monitoring units 4. 
U In a single market or a relatively small region, a central computer 7 receives broadcast 

M 1 5 signals directly and stores feature data continuously on its local disk 8. Assuming that on 
; 3 average a market will have 10 TV stations and 50 radio stations, the required storage is about 
173 Megabytes per day or 1210 Megabytes per week. Data older than one week is deleted. 
Obviously, as more sources are acquired through, e.g., satellite network feeds and cable 
television, the storage requirements increase. However, even with 500 broadcast sources the 
20 system needs only 10 Gigabytes of storage for a week of continuous storage. 

The recognition process requires that the central computer 7 locate time intervals in the 
stored feature blocks that are time aligned (within a few seconds) with the unknown feature 
packet. Since each portable monitoring unit 4 produces one packet per minute, the processing 
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load with 500 broadcast sources is 500 pattern matches per minute or about 8 matches per second 
for each portable monitoring unit. Assuming that there are 500 portable monitoring units in a 
market the system must perform about 4000 matches per second. 

When deployed on a large scale in many markets the overall system architecture is 
5 somewhat different as is illustrated in Figure 6. There are separate remote signal collection 
computers 33 installed in each city or market. The remote computers 33 record the broadcast 
sources in their particular markets as described above. In addition, they interrogate the portable 
monitoring units 34 in their area by modem 32 and download the collected feature packets. The 
signal collection computers 33 are connected to a central site by a wide area data communication 
u-10 network 35. The central computer site consists of a network 37 of computers 39 that can share 
- : • the pattern recognition processing load. The local network 37 is connected to the wide area 
f"„ network 35 to allow the central site computers 39 to access the collected feature packets and 
: broadcast feature data blocks. In operation, a central computer 39 downloads a day's worth of 
feature packets from a portable monitoring unit 34 that have been collected by one of the remote 
.U.U5 computers 33 using modems 32. Broadcast time segments that correspond to the packet times 
are then identified and transferred to the central site. The identification is then performed at the 
central site. Once an initial identification has been made, it is confirmed by matching subsequent 
packets with broadcast source features from the same channel as the previous recognition. This 
reduces the amount of data that must be transferred from the remote collection computer to the 
20 central site. This is based on the assumption that a listener will continue to listen (or stay tuned) 
to the same station for some amount of time. When a subsequent match fails, the remaining 
channels are downloaded for pattern recognition. This continues until a new match has been 
found. The system then reverts to the single-channel tracking mode. 
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The above process is repeated for all portable monitoring units 34 in all markets. In 
instances where markets overlap, feature packets from a particular portable unit can be compared 
with data from each market. This is accomplished by downloading the appropriate channel data 
from each market. In addition, signals that are available over a broad area such as satellite feeds, 
5 direct satellite broadcasts, etc. are collected directly at the central site using one or more satellite 
receivers 36. This includes many sources that are distributed over cable networks such as movie 
channels and other premium services. This reduces the number of sources that must be collected 
remotely (and redundantly) by the signal collection computers. 

An additional capability of this system configuration is the ability to match broadcast 
10 sources in different markets. This is useful where network affiliates may have several different 
selections of programming, 
f In the preferred embodiment of the portable monitoring unit shown in Figure 2 the audio 

: a signal received by small microphone 1 1 in a portable unit is amplified, lowpass filtered, and 
digitized by an analog to digital converter 13. The sample rate is 8 kilosamples per second, 
M 15 resulting in a Nyquist frequency of 4 kHz. To avoid alias distortion, an analog lowpass filter 12 
rejects frequencies greater than about 3.2 kHz. The analog to digital converter 13 sends the 
audio samples to a digital signal processing microprocessor 17 that performs the audio 
processing and feature extraction. The first step in this processing is spectrum analysis and 
partitioning of the audio spectrum into three frequency bands as shown in Figure 3. 
20 The frequency bands have been selected to contain approximately equal power on 

average. In one embodiment, the frequency bands are: 
Bandl: 50 Hz -500 Hz 
Band 2: 500 Hz -1500 Hz 
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Band 3: 1500 Hz- 3250 Hz 

It will be understood by those skilled in the art that other frequency bands may be used to 
implement the teachings of the present invention. 

The spectrum analysis is performed by periodically performing Fast Fourier Transforms 
5 (FFT's) on blocks of 64 samples. This produces spectra containing 32 frequency "bins". The 
power in each bin is found by squaring its magnitude. The power in each band is then computed 
as the sum of the power in the corresponding bins frequency. A magnitude value is then 
computed for each band by taking the square root of the integrated power. The mean value of 
each of these streams is then removed by using a recursive high-pass filter. The data rate and 
i= :;fl0 bandwidth must then be reduced. This is accomplished using polyphase decimating lowpass 
'% filters. Two filter stages are employed for each of the three feature streams. Each of these filters 

reduces the sample rate by a factor of five, resulting in a sample rate of 10 samples per second 
: .p (per stream) and a bandwidth of about 4 Hz. These are the audio data measurements that are 

used as features in the pattern recognition process. 
! 3;: - 15 A similar process is performed at the central computer site as shown in Figure 4. 

However, audio signals are obtained from direct connections to radio and television broadcast 
receivers. Since many audio sources must be collected simultaneously, a set of preamplifiers and 
analog lowpass filters 20 is included. The outputs of these filters are connected to a channel 
multiplexer 21 that switches sequentially between each audio signal and sends samples of these 
20 signals to the analog to digital converter 22. A digital signal processor 23 then operates on all of 
the audio time series waveforms to extract the features. 

To reduce the storage requirements in both the portable units and the central computers, 
the system employs mu-law compression of the feature data. This reduces the data by a factor 
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of two, compressing a 16-bit linear value to an eight bit logarithmic value. This maintains the 
Ml dynamic range while retaining adequate resolution for accurate correlation performance. 
The same feature processing is used in both the portable monitoring units and the central 
computers. However, the portable monitoring units capture brief segments of 64 feature samples 
5 at intervals of approximately one minute as triggered by a timer in the portable monitoring unit. 
Central computers record continuous streams of feature data. 

The portable monitoring unit is based on a low-power digital signal processor of the type 
that is frequently used in such applications as audio processing for digital cellular telephones. 
Most of the time this processor is in an idle or sleep condition to conserve battery power. 
1 10 However, an electronic timer operates continuously and activates the DSP at intervals of 
] approximately one minute. The DSP 17 collects about six seconds of audio from the analog to 
: ; , digital converter 13 and extracts audio features from the three frequency bands as described 
q previously. The value of the timer 15 is also read for use in time marking the collected signals. 

The portable monitoring unit also includes a rechargeable battery 19 and a docking station data 
: 45 interface 18. 

- In addition to the features that are collected, the total audio power present in the six- 
second block is computed to determine if an audio signal is present. The audio signal power is 
then compared with an activation threshold. If the power is less than the threshold the collected 
data are discarded, and the DSP 17 returns to the inactive state until the next sampling interval. 
20 This avoids the need to store data blocks that are collected while the user is asleep or in a quiet 
environment. If the audio power is greater than the threshold, then the data block is stored in a 
non- volatile memory 16. 
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Feature data to be stored are organized as 64 samples of each of the three feature streams. 
These data are first mu-law compressed from 16 bit linear samples to 8 bit logarithmic samples. 
The resulting data packets therefore contain 192 data bytes. The data packets also contain a 
four-byte unit identification code and a four-byte timer value for a total of 200 bytes per packet. 
5 The data packets are stored in a non- volatile flash memory 16 so that they will be retained when 
power is not applied. After storing the data packet, the unit returns to the sleep-state until the 
next sampling interval. This procedure is illustrated in flow-chart form in Figure 7. 

Figure 5 is a block diagram of the portable unit docking station 10. The docking station 
includes a data interface 28 to the portable unit 4 and a dialup modem 29 that is used to 
: ;1 0 communicate with modems 9 that are connected to the central computer 7. An AC power supply 
■ I- 31 supplies power to the docking station and also powers a battery charger 30 that is used to 
recharge the battery 1 9 in the portable monitoring unit 4. 

When the portable monitoring unit 4 is in its docking station 10 and communicates with a 
central computer 7, packets are transferred in reverse order. That is, the newest data packets are 
; 15 transferred first, proceeding backwards in time. The central computer continues to transfer 
:i ; packets until it encounters a packet that has been previously transferred. 

Each portable monitoring unit 4 optionally includes a motion detector or sensor (not 
shown) that detects whether or not the device is actually been worn or carried by the user. Data 
indicating movement of the device is then stored (for later downloading and analysis) along with 
20 the audio feature information described above. In one embodiment, audio feature information is 
discarded or ignored in the survey process if the output of the motion detector indicated that the 
device 4 was not actually been worn or carried during a significant period of time when the audio 
information was being recorded. 
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Each portable monitoring unit 4 also optionally includes a receiver (not shown) used for 
determining the position of the unit (e.g., a GPS receiver, a cellular telephone receiver, etc.). 
Data indicating position of the device is then stored (for later downloading and analysis) along 
with the audio feature information described above. In one embodiment, the downloaded 
5 position information is used by the central computer to determine which signal collection 
station's features to access for comparison. 

In contrast with the portable monitoring units that sample the audio environment 
periodically, the central computer must operate continuously, storing feature data blocks from 
many audio sources. The central computer then compares feature packets that have been 
QO downloaded from the portable units with sections of audio files that occurred at the same date 
and time. There are three separate processes operating in the data collection and storage aspect 
of central computer operation. The first of these is the collection and storage of digitized audio 
; j j data and storage on the disks 8 of the central computer. The second task is the extraction of 
M feature data and the storage of time-tagged blocks of feature data on the disk. The third task is 
h45 the automatic deletion of feature files that are old enough that they can be considered to be 
; =|;f irrelevant (one week). These processes are illustrated in Figure 8. 

Audio signals may be received from any of a number of sources including broadcast 
radio and television, satellite distribution systems, subscription services, and the internet. 
Digitized audio signals are stored for a relatively short time (along with time markers) on the 
20 central computer pending processing to extract the audio features. It is frequently beneficial to 
directly compute the features in real-time using special purpose DSP boards that combine analog 
to digital conversion with feature extraction. In this case the temporary storage of raw audio is 
greatly reduced. 
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The audio feature blocks are computed in the same manner as for the portable monitoring 
units. The central computer system 7 selects a block of audio data from a particular channel or 
source and performs a spectrum analysis. It then integrates the power in each of three frequency 
bands and outputs a measurement. Sequences of these measurements are lowpass filtered and 
5 decimated to produce a feature sample rate of 10 samples per second for each of the three bands. 
Mu-law compression is used to produce logarithmic amplitude measurements of one byte each, 
reducing the storage requirements. Feature samples are gathered into blocks, labeled with their 
source and time, and stored on the disk. This process is repeated for all available data blocks 
from all channels. The system then waits for more audio data to become available. 
■ -iO In order to control the requirement for disk file storage, feature files are labeled with their 

: t~ date and time of initiation. For example, a file name may be automatically constructed that 
:7 contains the day of the week and hour of the day. An independent task then scans the feature 
. storage areas and deletes files that are older than a specified amount. While the system expects 
y s to interrogate portable monitoring units on a daily basis and to compare their collected features 
h -15 with the data base every day, there will be cases where it will not be possible to interrogate some 
of the portable units for several days. Therefore, feature data are retained at the central computer 
site for about a week. After that, the results will no longer be useful. 

When the central computer 7 compares audio feature blocks stored on its own disk drive 
8 with those from a portable monitoring unit 4, it must match its time markers with those 
20 transferred from the portable monitoring unit. This reduces the amount of searching that must be 
done, improving the speed and accuracy of the processing. 

Each portable monitoring unit 4 contains its own internal clock 15. To avoid the need to 
set this clock or maintain any specific calibration, a simple 32-bit counter is used that is 
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incremented at a 10 Hz rate. This 10 Hz signal is derived from an accurate crystal oscillator. In 
fact, the absolute accuracy of this oscillator is not very important. What is important is the 
stability of the oscillator. The central site interrogates each portable monitoring unit at intervals 
of between one day and once per week. As part of this procedure the central site reads the 
5 current value of the counter in the portable monitoring unit. It will also note its own time count 
and store both values. To synchronize time the system subtracts the time count that was read 
from the portable unit during the previous interrogation from the current value. Similarly, the 
system computes the number of counts that occurred at the central site (the official time) by 
subtracting its stored counter value from the current counter value. If the frequencies are the 
1 ;;|0 same, the same number of counts will have transpired over the same time interval (6.048 Million 
1- counts per week). In this case the portable unit 4 can be synchronized to the central computer 7 
: : ; by adding the difference between the starting counts to the time markers that identify each audio 

feature measurement packet. This is the simplest case. 
h= The typical case is where the oscillators are running at slightly different frequencies. It 

Ml 5 is still necessary to align the starting counter values, but the system must also compute a scale 
! =i;f factor and apply it to time markers received from the portable monitoring unit. This scale factor 
is computed by dividing the number of counts from the central computer by the number of 
counts from the portable unit that occurred over the same time interval. The first order (linear) 
time synchronization requires computation of an offset and a scale factor to be applied to the 
20 time marks from the portable monitoring unit. 

Compute Offset Off = S c 

Compute Central Counts C c = E c 

Compute Portable Counts C p = E p 



-s P 
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Compute Scale Factor Scl = C c /C p 

Time markers can then be converted from the portable monitoring unit to the central 
computer frame of reference: 

Convert Time Marker T c = (T p + Off) * Scl 

5 The remaining concern is short-term drift of the oscillator in the portable monitoring unit. 

This is primarily due to temperature changes. The goal is to stay within one second of the 
linearly interpolated time. The worst timing errors occur when the frequency deviates in one 
direction and then in the opposite direction. However, it has been determined that stability will 
be adequate over realistic temperature ranges. 
: ; flO The audience survey system includes pattern recognition algorithms that determine which 

'% of many possible audio sources was captured by a particular portable monitoring unit 4 at a 
n certain time. To accomplish this with reasonable hardware cost, the central computers 7 
y;i preferably employ high performance PC's 25 that have been augmented by digital signal 

processors 26 that have been optimized to perform functions such as correlations and vector 
i =1 * 15 operations. Figure 9 summarizes the signal recognition procedure. 

s *j;f As discussed previously, it is important to synchronize the time markers received from 

the portable monitoring units 4 with the time tags applied to feature blocks stored on the central 
computer systems 7. Once this has been done, the system should be able to find stored feature 
blocks that are within about one second from the feature packets received from the portable 
20 units. The tolerance for time alignment is about +/- 3 seconds, leaving some room to deal with 
unusual situations. Additionally, the system can search for pattern matches outside of the 
tolerance window, but this slows down the processing. In cases where pattern matches are not 
found for a particular portable unit, the central computer can repeat all of the pattern matches 
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using an expanded search window. Then when matches are found, their times of occurrence can 
be used as checkpoints to update the timing information. However, the need to resort to these 
measures may indicate a malfunction of the portable monitoring unit or its exposure to 
environmental extremes. 

5 The pattern recognition process involves computing the degree of match with reference 

patterns derived from features of each of the sources. As shown in Figure 9, this degree of match 
is measured as a weighted Euclidean distance in three-dimensional space. The distance metric 
indicates a perfect match as a distance of zero. Small distances indicate a closer match than large 
distances. Therefore, the system must find the source that produces the smallest distance to the 
□10 unknown feature packet. This distance is then compared with a threshold value. If the distance 
^ is below the threshold, the system will report that the unknown packet matches the 
; : v% corresponding source and record the source identification. If the minimum distance is greater 
than the threshold, the system presumes that the unknown feature packet does not match any of 
the sources and record that the source is unknown, 
h 1 1 5 The basic pattern matching procedure is illustrated in Figure 1 0. Feature packets from a 

portable monitoring unit 4 contain 64 samples from each of the three bands. These must first be 
mu-law decompressed to produce 16 bit linear values. Each of the three feature waveforms is 
then normalized by dividing each value by the standard deviation (square root of power) 
computed over the three signals. This corrects for the audio volume to which the portable unit 
20 was exposed when the feature packet was collected. Each of the three normalized waveforms is 
then padded with a block of zeroes to a total length of 128 samples per feature band. This is 
necessary to take advantage of a fast correlation algorithm based on the FFT. 
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The system then locates a block of samples consisting of 128 samples of each feature as 
determined by the time alignment calculation. This will include the time offset needed to assure 
that the needed three second margins are present at the beginning and end of the expected 
location of the unknown packet. Next, the system calculates the cross-correlation functions 
5 between each of the three waveforms of the unknown feature packet and the corresponding 
source waveforms. In the fast correlation algorithm this requires that both the unknown and the 
reference source waveforms are transformed to the frequency domain using a fast Fourier 
transform. The system then performs a conjugate vector cross-product of the resulting complex 
spectra and then performs an inverse fast Fourier transform on the result. The resulting 

OlO correlation functions are then normalized by the sliding standard deviation of each computed 

: % over a 64 sample window. 

; ^ Each of the three correlation functions representing the three frequency bands have a 

maximum value of one for a perfect match to zero for no correlation to minus one for an exact 

| S!! i s opposite. Each of the correlation values is converted to a distance component by subtracting it 

h* 15 from one. The Euclidean distance is preferably defined as set forth in equation (1) below as the 

! =W square root of the sum of the squares of the individual components: 

D=[(l<vd 2 + (l-cv 2 f + (l-cv 3 ) 2 ] m (1) 
This results in a single number that measures how well a feature packet matches the reference (or 
source) pattern, combining the individual distances as though they were based on measurements 
20 taken in three dimensional space. However, by virtue of normalizing the feature waveforms, 
each component makes an equal contribution to the overall distance regardless of the relative 
amplitudes of the audio in the three bands. In one embodiment, the present invention aims to 
avoid situations where background noise in an otherwise quiet band disturbs the contributions of 
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frequency bands containing useful signal energy. Therefore, the system reintroduces relative 
amplitude information to the distance calculation by weighting each component by the standard 
deviations computed from the reference pattern as shown in equation (2) below. This must be 
normalized by the total magnitude of the signal: 
5 Dw=[((stdj) *(l<vdf+((stdj *(l~cv 2 )) 2 +((std 3 ) ^(l-cvs)) 2 f 2 /[(std l ) 2 ^std 2 f (2) 
The sequence of operations can be rearranged to combine some steps and eliminate others. The 
resulting weighted Euclidean distance automatically adapts to the relative amplitudes of the 
frequency bands and will tend to reduce the effects of broadband noise that is present at the 
portable unit and not at the source. 
l:: ::flO A variation of the weighted Euclidean distance involves integrating or averaging 

t: successive distances calculated from a sequence of feature packets received from a portable unit 
i n as shown in Figure 1 1 . In this procedure, the weighted distance is computed as above for the 
J;! first packet. A second packet is then obtained and precisely aligned with feature blocks from the 

same source in the central computer. Again, the weighted Euclidean distance is calculated. If 
i 15 the two packets are from the same source, the minimum distance will occur at the same relative 
time delay in the distance calculation. For each of the 64 time delays in the distance array for a 
particular source the system computes a recursive update of the distance where the averaged 
distance is decayed slightly by multiplying it by a coefficient k that is less than one. The newly 
calculated distance is then scaled by multiplying it by (1-k) and adding it to the average distance. 
20 For a particular time delay value within the distance array the update procedure can be expressed 
as shown in equation (3) below: 

Dw(n)=k*D w (n-l) + (l-k)*D w (n) (3) 
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Note that the bold notation IV indicates the averaged value of the distance calculation, (n) refers 
to the current update cycle, and (n-1) refers to the previous update cycle. This process is 
repeated on subsequent blocks, recursively integrating more signal energy. The result of this is 
an improved signal-to-noise ratio in the distance calculation that reduces the probability of false 
detection. 

The decision rule for this process is the same as for the un-averaged case. The minimum 
averaged distance from all sources is first found. This is compared with a distance threshold. If 
the minimum distance is less than the threshold, a detection has occurred and the source 
identification is recorded. Otherwise the system reports that the source is unknown. 

The previous description of the preferred embodiments is provided to enable any person 
skilled in the art to make and use the present invention. The various modifications to these 
embodiments will be readily apparent to those skilled in the art, and the generic principles 
defined herein may be applied to other embodiments without the use of the inventive faculty. 
Thus, the present invention is not intended to be limited to the embodiments shown herein but is 
to be accorded the widest scope consistent with the principles and novel features disclosed 
herein. 



What is claimed is: 

1 . An audience survey system, comprising: 

(A) a plurality of portable monitoring units that are assigned to users that are members 
of an audience panel, wherein each portable monitoring unit records information representative 

5 of free field audio signals received by the portable monitoring unit, and the information 

representative of the free field audio signals includes information representing content of free 
field audio signals and time stamp information indicating when the free field audio signals were 
received by the portable monitoring unit; 

(B) a central broadcast collection facility that records information representative of 
1 0 audio signals transmitted from a plurality of sources, wherein for each audio signal the 

information recorded by the central broadcast collection facility includes information 
representing content of the audio signal, time stamp information indicating when the audio signal 
was received by the central broadcast collection facility, and source information indicating a 
source that transmitted the audio signal; and 

15 (C) a computer that identifies the source selected by each user of a portable 

monitoring unit during each of a plurality of different time periods in accordance with the 
information recorded by the portable monitoring units and the information recorded by the 
central broadcast collection facility. 

2. The system of claim 1 , wherein each portable monitoring unit periodically records 
20 the information representative of the free field audio signals received by the portable monitoring 
unit. 
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3. The system of claim 2, wherein the central broadcast collection facility 
continuously records the information representative of audio signals broadcast from the plurality 
of sources. 

4. The system of claim 3, wherein the computer is coupled to the central broadcast 
5 collection facility, said system further comprising: 

(D) a plurality of docking stations each of which periodically downloads the 
information recorded by a portable monitoring unit to the computer. 

5 . The system of claim 4, wherein each of the docking stations includes a modem for 
communicating with the computer, and a charger that charges a battery in a portable monitoring 

1 0 unit when the portable monitoring unit is positioned in the docking station. 

6. The system of claim 1 , wherein each portable monitoring unit includes a 
microphone that receives free field audio signals associated with a source selected by the user of 
the portable monitoring unit. 

7. The system of claim 1 , wherein each portable monitoring unit is worn or carried 
15 by a user. 

8. The system of claim 1 , wherein the information representative of the free field 
audio signals recorded by each portable monitoring unit includes a digitally compressed version 
of content associated with free field audio signals received by the portable monitoring unit. 
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9. The system of claim 8, wherein the information recorded by the central broadcast 
collection facility includes a digitally compressed version of content associated with the audio 
signals received by the central broadcast collection facility. 

10. The method of claim 1, wherein the central broadcast collection facility and the 
5 computer that identifies the source selected by each user of a portable monitoring unit during 

each of a plurality of different time periods are implemented using a common host computer. 

11. A method for performing an audience survey, comprising the steps of: 

(A) providing a plurality of portable monitoring units to users that are members of an 
ji audience panel, wherein each portable monitoring unit records information representative of free 
p; 10 field audio signals received by the portable monitoring unit, and the information representative 

of the free field audio signals includes information representing content of free field audio 
y signals and time stamp information indicating when the free field audio signals were received by 

the portable monitoring unit; 

i=i (B) recording, at a central broadcast collection facility, information representative of 

15 audio signals broadcast from a plurality of sources, wherein for each audio signal the information 
recorded by the central broadcast collection facility includes information representing content of 
the audio signal, time stamp information indicating when the audio signal was received by the 
central broadcast collection facility, and source information indicating a source that transmitted 
the audio signal; and 

20 (C) identifying the source selected by each user of a portable monitoring unit during 

each of a plurality of different time periods in accordance with the information recorded by the 
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portable monitoring units and the information recorded by the central broadcast collection 
facility. 

12. A method for forming compressed audio signals, comprising the steps of: 

(A) acquiring an audio signal from a microphone or a broadcast receiver; 

(B) for each of a plurality of time periods, measuring a power level associated with 
the acquired audio signal in each of at least three frequency bands; 

(C) constructing time domain audio feature waveforms from the results of step (B), 
wherein each time domain audio feature waveform represents power levels measured in one of 
the at least three frequency bands over the plurality of time periods; and 

(D) forming logarithmically compressed audio signals representative of the acquired 
audio signal by applying mu-law compression to the results of step (C). 

13. The method of claim 12, further comprising: 

(E) constructing packets of feature waveforms from the results of step (D), wherein 
each packet of feature waveforms is representative of several contiguous seconds of the acquired 
audio signal 

14. The method of claim 13, wherein step (E) further comprises applying a time 
marker to each packet representative of a time when the audio signal was acquired from the 
microphone or the broadcast receiver. 

15. The method of claim 12, further comprising: 
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(E) constructing continuous feature waveforms from the results of step (D), and 
storing the continuous feature waveforms in a computer. 

16. The method of claim 15, wherein step (E) further comprises periodically applying 
time markers to segments of the continuous feature waveforms, each time marker being 

5 representative of a time when the audio signal was acquired from the microphone or the 
broadcast receiver. 

17. The method of claim 16, further comprising the step of: 

(F) periodically deleting from the computer segments of the continuous feature 
Ji waveforms having time markers that are older than a specified limit. 

ul 0 1 8. A method for synchronizing time between a portable data collection unit and a 

UJ host computer that receives downloaded information from the portable data collection unit, 
comprising the steps of: 

^ (A) marking information recorded in the portable data collection unit with time 

'"!;; markers that are obtained from an output of a first counter in the portable data collection unit; 

1 5 (B) downloading the time marked information from the portable data collection unit 

to the host computer at a time T and storing, in the host computer, an output of the first counter 
and an output of a second counter in the host computer at the time T; and 

(C) adjusting the time markers in the time marked information in accordance with the 
output of the first counter at time T, the output of the second counter at time T, and any 
20 frequency difference between a frequency of the first counter and a frequency of the second 
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counter, thereby synchronizing the time markers in the time marked information to the second 
counter in the host computer. 

19. The method of claim 18, wherein the frequency difference between the frequency 
of the first counter and the frequency of the second counter is zero. 

5 20. The method of claim 18, wherein the output of the second counter corresponds to 

an absolute system time. 

2 1 . The method of claim 1 8, wherein steps (B) and (C) comprise the following steps: 

(B) interrogating the portable data collection unit with the host computer at a first 
j;: time Tl and storing, in the host computer, an output of the first counter and an output of a second 
M10 counter in the host computer at the first time Tl ; and, after the interrogating step, downloading 
UJ the time marked information from the portable data collection unit to the host computer at a 

second time T2 and storing, in the host computer, an output of the first counter and an output of a 

second counter in the host computer at the second time T2; and 

" (C) adjusting the time markers in the time marked information in accordance with the 

15 output of the first counter at times Tl and T2 and the output of the second counter at times Tl 
and T2, thereby synchronizing the time markers in the time marked information to the second 
counter in the host computer. 

22. The method of claim 21, wherein step (C) further comprises the steps of: 

(i) determining a first elapsed time value by comparing the output of the first 
20 counter at the first time Tl with the output of the first counter at the second time T2; 
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(ii) determining a second elapsed time value by comparing the output of the 
second counter at the first time Tl with the output of the second counter at the second time T2; 

(iii) determining a scale factor (Scl) in accordance with the first elapsed time 
value and the second elapsed time value; and 

5 (iv) adjusting each time marker (T p ) in the time marked information to a time 

value (T c ) that is synchronized with the second counter in the host computer in accordance with 
the following equation: 

T c = (T p + Off)*Scl 

, where Off corresponds to an offset between the first counter and the second counter. 

i , : ;10 23, The method of claim 22, wherein Off has a value corresponding to a difference 

between the output of the second counter at the first time Tl and the output of the first counter at 
f " the first time Tl. 

ia !j; 24, An apparatus for synchronizing time between a portable data collection unit and a 

host computer that receives downloaded information from the portable data collection unit, 
1 5 wherein information is recorded in the portable data collection unit with time markers that are 
obtained from an output of a first counter in the portable data collection unit, comprising: 

a host computer that downloads the time marked information from the portable data 
collection unit to the host computer at a time T, stores an output of the first counter and an output 
of a second counter in the host computer at the time T, and adjusts the time markers in the time 
20 marked information in accordance with the output of the first counter at time T, the output of the 
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second counter at time T, and any frequency difference between a frequency of the first counter 
and a frequency of the second counter, thereby synchronizing the time markers in the time 
marked information to the second counter in the host computer. 

25. A portable data collection unit that operates in either a sleep mode or an active 
5 mode, comprising: 

(A) a microphone that receives free field audio signals that are audible to a user 
proximate the portable data collection unit; 

(B) a processor that periodically places the portable data collection unit into the active 
J mode for a predetermined period of time after which the controller places the portable data 

= "1 0 collection unit into the sleep mode; 

wherein the processor is coupled to an output of the microphone and compares the output 
i . of the microphone to a threshold when the portable data collection unit is in the active mode; and 

wherein the processor stores data representative of the free field signals only during 
A periods when the portable data collection unit is in the active mode and the output of the 
1 5 microphone exceeds the threshold. 

26. A system for periodically transferring information from portable monitoring units 
to a central computer, comprising; 

(A) a plurality of portable monitoring units that are assigned to users, wherein each 
portable monitoring unit records information representative of free field audio signals received 
20 by the portable monitoring unit; 
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(B) a plurality of docking stations each of which receives a portable monitoring unit 
when the portable monitoring unit is not being worn by one of the users, wherein each of the 
docking stations includes a modem; and 

5 (C) a central information collection facility that periodically places a call to the 

modem in each of the docking stations, wherein the central information collection facility 
downloads information stored in a given portable monitoring unit if the given portable 
monitoring unit is positioned in a docking station when the central information collection facility 
places the call to the docking station. 

4-10 27, A method for correlating a first packet of feature waveforms from an unknown 

! =l { source with a second packet of feature waveforms from a known source in order to associate a 
known source with the first packet of feature waveforms, comprising the steps of: 

| ss:L (A) determining at least first, second and third correlation values (cv u CV2, cv 3 ) by 

ji correlating features from the first and second packets, wherein the first correlation value (cvi) is 
15 determined by correlating features associated with a first frequency band from the first and 
second packets, the second correlation value (cv 2 ) is determined by correlating features 
associated with a second frequency band from the first and second packets, and the third 
correlation value (CV3) is determined by correlating features associated with a third frequency 
band from the first and second packets; 

20 (B) computing a first weighting value in accordance with the features from the second 

packet associated with the first frequency band, a second weighting value in accordance with the 
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features from the second packet associated with the second frequency band, and a third 
weighting value in accordance with the features from second packet associated with the third 
frequency band; 

(C) computing a weighted Euclidean distance value (D w ) representative of differences 
5 between the first and second packets from the first, second and third correlation values and the 

first, second and third weighting values; and 

(D) associating the first frequency packet with the known source in accordance with 
the weighted Euclidean distance value (D w ). 

28. The method of claim 27, wherein the first weighting value corresponds to a 
11 0 standard deviation (stdi) of the features from the second packet associated with the first 

frequency band, the second weighting value corresponds to a standard deviation (std2) of the 
features from the second packet associated with the second frequency band, and the third 
weighting value corresponds to a standard deviation (std 3 ) of the features from the second packet 
associated with the third frequency band. 

1 5 29. The method of claim 28, wherein the weighted Euclidean distance value (D w ) is 

determined in accordance with the following equation: 

Dw = [((stdO*(l-cvO) 2 + ((std 2 )*(l-cv 2 )) 2 + ((std 3 )*(l-cv 3 )) 2 ] 1/2 / [(std 1 ) 2 +(std 2 ) 2 +(std 3 ) 2 ] l/2 

30. The method of claim 27, wherein step (D) comprises: 

(D) associating the first frequency packet with the known source if the weighted 
20 Euclidean distance value (D w ) is less than a threshold. 
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31. A method for correlating a packet of feature waveforms from an unknown source 
with a packet of feature waveforms from a known source in order to associate a known source 
with the packet of feature waveforms from the unknown source, comprising the steps of: 

(A) determining at least first, second and third correlation values by correlating 
5 features from first and second packets, wherein the first correlation value is determined by 

correlating features associated with a first frequency band from the first and second packets, the 
second correlation value is determined by correlating features associated with a second 
frequency band from the first and second packets, and the third correlation value is determined 
by correlating features associated with a third frequency band from the first and second packets; 

K 0 (B) computing a Euclidean distance value (D(n- 1 )) representative of differences 

; 5 .;] between the first and second packets from the first, second and third correlation values; 

(C) determining at least fourth, fifth and sixth correlation values by correlating 

i. features from third and fourth packets, wherein the fourth correlation value is determined by 
correlating features associated with the first frequency band from the third and fourth packets, 

==)■'; 1 5 the fifth correlation value is determined by correlating features associated with the second 

frequency band from the third and fourth packets, and the sixth correlation value is determined 
by correlating features associated with the third frequency band from the third and fourth 
packets; 

(D) computing a Euclidean distance value (D(n)) representative of differences 
20 between the third and fourth packets from the fourth, fifth and sixth correlation values; 
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(E) updating the Euclidean distance value (D(n)) using the Euclidean distance value 
(D(n-1)); and 

(F) associating the third packet with the known source in accordance with the updated 
Euclidean distance value (D(n)). 

5 32. The method of claim 3 1 , wherein the second and fourth packets are known a 

priori to represent signals broadcast from the known source. 

33. The method of claim 32, wherein the third packet is positioned immediately after 
the first packet in a sequence of packets of feature waveforms. 

v[ 34. The method of claim 33, wherein the fourth packet is positioned immediately 

f ; 1 0 after the second packet in a sequence of packets of feature waveforms. 

35. The method of claim 34, wherein the updated the Euclidean distance value (D(n)) 
[ : is determined in step (E) in accordance with the following equation: 

D(n) = k * D(n-1) + (1-k) * D(n) 

where k is a coefficient that is less than 1. 

15 36. The method of claim 31, wherein step (F) comprises: 

(F) associating the third frequency packet with the known source if the updated 
Euclidean distance value (D(n)) is less than a threshold. 
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ABSTRACT 

A system and method are disclosed for performing audience surveys of broadcast audio 
from radio and television. A small body-worn portable collection unit samples the audio 
environment of the survey member and stores highly compressed features of the audio 

5 programming. A central computer simultaneously collects the audio outputs from a number of 
radio and television receivers representing the possible selections that a survey member may 
choose. On a regular schedule the central computer interrogates the portable units used in the 
survey and transfers the captured audio feature samples. The central computer then applies a 
feature pattern recognition technique to identify which radio or television station the survey 

0 member was listening to at various times of day. This information is then used to estimate the 
popularity of the various broadcast stations. 



-37- 




E 

(D 

-4— « 
(/) 

CO 

<D 

3 
Q. 

E 
o 
O 

"co 

"c 

CD 

O 

CD 
i_ 

D) 



-O Q- 

3 
< 



c cj2 

fc co -5 = 

CO 5 UL 

CD < 



to D 
CO Q. 



START 



Wake Up Each 
Minute 
and Collect an Audio 
Sample of Six 
Seconds Duration 



Extract Audio Feature 
Waveforms from 
Each 
of Three Frequency 
Bands 




Mu-Law Compress 
Audio Feature 
Waveform Data 



Sleep Until Next 
Sample Point 



Construct Time- 
Tagged Feature 
Packet 



Store Feature Packet 
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re 8: Central Computer Feature Extraction and Storage 
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Figure 9: Unknown Packet Identification Procedure 
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